Testing is commonplace in today's society. Testing is often required to obtain access to schools, jobs, or organizations. While tests are easy to administer to the general population, it is important that tests are also designed to be as accessible as possible to all individuals, including individuals with disabilities or those having other characteristics that separate them from typical test takers. Exemplary features that could be considered when designing and administering assessments include large print and enlarged computer fonts, Braille (hard copy or refreshable), audio (also called “readaloud”: synthesized speech, live reader, prerecorded audio), raised-line (tactile drawings, sometimes with Braille labels), visual highlighting on a computer display as text is read aloud, color/contrast modification, spelling and grammar checkers, dictionaries, and extended testing time. Such accessibility features and formats may be allowed based on prior approval (accommodations) or may be allowed to virtually anyone who needs or desires the feature (“universal design” features).
Synthesized speech for computers can be provided in at least two ways: screen readers and self-voicing applications. Screen reader programs are commercial text-to-speech software packages that allow a person who is blind, or has certain other specific disabilities, to interact with a variety of other software, such as word processors, spreadsheets and Web browsers. The most popular screen reader programs are very feature rich but often require considerable practice for effective use. Self-voicing applications are developed to provide built-in speech synthesis and therefore do not rely on commercial screen reader technology.
A refreshable Braille display has hardware that raises and lowers Braille dot patterns on command from a computer. Users can read the Braille dot patterns by moving their fingertips across the Braille panel, much as they would read the Braille dot patterns on paper. Refreshable Braille displays enable access to computers by individuals who are deaf-blind.
Unfortunately, accessibility features that may be useful in overcoming accessibility barriers can, in some instances, invalidate test results. For example, a person with a spelling disability (dysorthographia) could argue that their use of spelling checker software would help overcome an accessibility barrier on educational tests that involve writing. If a test is intended to measure spelling ability, though, such an accommodation would tend to invalidate the test results by providing an unfair advantage to such a person.
A committee of the National Research Council (NRC) that examined accommodation policies for the United States' National Assessment of Educational Progress (NAEP) and other large-scale assessments noted that determining which accommodation is right for a particular circumstance is difficult. The accommodation must at the same time be directly related to the disability or lack of fluency for which it is to compensate and be independent of the constructs on which the test taker is to be assessed. An example of this difficulty concerns the use of a “readaloud” accessibility feature. The readaloud capability may enable individuals with visual impairments and certain learning disabilities to take a particular assessment. The readaloud accommodation is allowed on several state assessments of reading. However, the United States' National Assessment of Educational Progress (NAEP) does not allow test content to be read aloud on its reading assessment. The NAEP considers that accommodation to be incompatible with the nature of the proficiency being measured (reading). This discrepancy underscores the issue of identifying principles, criteria and/or procedures for determining which accessibility features can be offered without compromising the validity of the scores.
A committee of the National Research Council that examined accommodation policies for the NAEP and other large-scale assessments stated that existing research does not provide definitive evidence about which procedures will produce the most valid estimates of performance for students with disabilities and English language learners. The investigation that led to this conclusion included a review of research on the “interaction hypothesis,” which posits that a valid accommodation will improve scores for students who need the accommodation but not for the students who do not need the accommodation; studies based on the interaction hypothesis have not provided evidence sufficient for determining the validity of an accommodation. The NRC committee concluded that by urging sponsors of large-scale assessment programs to identify the inferences that they intend to make from their assessment results and to embark on a research agenda that is guided by the claims and counterclaims for intended uses of results in the validation argument they have articulated.
Evidence Centered Assessment Design (ECD) was originally formulated at Educational Testing Service. ECD seeks to make the evidentiary argument embodied in assessment systems more explicit and clarify assessment design decisions. Structures and supporting rationales for the evidentiary argument of an assessment are designed to make the argument easier to examine, share and refine.
Recently, ECD researchers have sought to determine how validity is impacted by accessibility features provided to students with disabilities and English language learners. These extensions include structures that represent and explore a core validity issue, specifically, the alignment between (a) what one intends to measure and (b) what one is actually measuring in operational settings. In particular, alignment favors validity, and misalignment favors invalidity.
A need exists for methods of applying this approach to enable those having characteristics differing from the general population to have adequate access to tests.
A need exists for a conceptual framework that clarifies which features to provide with accessible testing systems.
A still further need exists for a conceptual framework that clarifies how much control to allow test takers in the use of such features.
A further need exists for enabling test designers and testing system designers to build accessibility into their designs from the earliest stages, thereby overcoming barriers faced by individuals with disabilities and English language learners.
The present invention is directed to solving one or more of the above-listed problems.